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Abstract Carol Weiss did much to enhance the role of 
evaluation in her writings. Her work shows evaluators what 
affects their roles as they evaluate programs. Furthermore, 
her theory of change spells out the complexities involved in 
program evaluation. There are various processes involved in 
the evaluation of programs. The paper looks at some of the 
aspects that inform this process. Effective evaluation of 
programs can lead to improvement of programs and good 
delivery of projects in future. It is Weiss who stated that 
evaluation should have an influence on policy and practice. 
Amongst other themes explored here are the politics of 
evaluation as well as the crucial role of the evaluator within 
the complexity of evaluation of programs. 
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1. Introduction 

This article explores aspects of Carol Weiss’s theory of 
change. Hanberger [1] cites several authors including Weiss 
who state that monitoring and evaluation are crucial in 
today’s society; moreover, these authors support democratic 
governance and promote accountability as well as program 
improvement. Weiss [2] defines the purpose of evaluation as 
a process “to measure the effects of a program against the 
goals it set out to accomplish as a means of contributing to 
subsequent decision making about the program and 
improving future programming”. Her interest in evaluation 
was always to examine the goals that the program itself 
promulgated. Evaluation needs to influence decision making 
as it ensures that the evaluation will be able to improve future 
programs. Weiss’ theory [3] has also been influenced by the 
political situation as all programs tend to be influenced by a 
certain political atmosphere. This then means that the 
political context affects the work of evaluators; evaluators 
are pressured by political influences. 

In fact, Weiss and Alkin [4] contend that there are three 
principal ways in which politics encroach on program 
education: 

i. Programs are created and maintained by political 


forces: 

ii. (Higher echelons of government, which make 
decisions about programs are embedded in politics; 
and 

iii. The very act of evaluation has political connotations. 
Weiss is aware of the challenges that the evaluation 
process constantly encounters. Sometimes it might not 
be easy for an evaluator to have appropriate tools and 
techniques to understand fully what is going on (Weiss 
1998). 

Furthermore, Weiss [5] argues: Theory-based evaluation 
is one approach that has a great deal of promise. But trying to 
use theory-based evaluation is difficult when programs do 
not have any explicit- or even implicit-theories, when 
programs are amorphous, or when they shift significantly 
over time. Evaluators cannot rely solely on their expertise in 
research methodology any longer. They have to understand 
the program field.The above shows the need to plan for 
evaluation thoroughly. 

Weiss stresses the need for evaluators to understand a 
program and how it works well. This author speaks of what 
she calls evaluation appreciation. She argues that for 
practitioners to conduct effective evaluation they need to 
understand what evaluation is all about as well as what it 
takes to conduct a good study. Furthermore, evaluators need 
to know what to do with evaluation results. Hanberger [1] 
highlights that there is interplay between evaluation and 
governance and that all evaluation systems are crucial in 
democratic governance. 

Whilst this paper focuses on the work of Carol Weiss one 
should be aware of the broad themes agreed upon by 
evaluators. Effective evaluations lead to the improvement of 
programs and good delivery of projects in future. Of crucial 
importance is that evaluation should have an influence on 
policy and practice [6]. The latter promulgation has 
influenced a number of evaluation experts. However, [6] 
argue that there is general consensus and that evaluation is 
mainly used in three ways; firstly, it is used to give direction 
to policy and practice; secondly, it is used to justify 
pre-existing preferences and actions and finally, to provide 
new generalizations, ideas and concepts. All those involved 
in social policymaking need effective evaluation strategies 
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based on sound theory. Programs are usually very complex 
and frequently need effective program theory for evaluation. 
However, there are considerable challenges to program 
theory and not all evaluators are convinced it is useful [7]. 
Furthermore, [7] argues that sometimes referred to as 
program logic or theory of change it points to a variety of 
ways of developing a causal modal linking program inputs to 
a chain of observed outcomes and then using this model to 
guide evaluation. Various simple logic models explicate the 
theory of change espoused by experts such as Rogers, 
Funnell and Weiss. Rogers, Huebner and Hacsi [8] consider 
program theory of evaluation to have two essential 
components, one perceptual and the other empirical. It is 
crucial to utilize theories well when conducting evaluations. 
Rogers et al. [8] assert: 

In some evaluations, the program theory has been 
developed largely by the evaluator, based on a review of 
research literature on similar programs or relevant causal 
mechanisms, through discussions with key informants 
through a review of program documentation, or through 
observation of the program itself (Lipsey and Pollard, 1989). 
In other evaluation, the program theory has been developed 
primarily by those associated with the program, often 
through a group process. 

When employing any permission to evaluate the 
practitioners are interested in exploring the ultimate 
outcomes. The outcomes will eventually impact on policy. 
This paper explores Weiss understand the Theoiy of Change 

2. The Theory of Change 

As seen in the discussions above, Weiss is linked to this 
concept which she has used widely in her work as an 
evaluator [9]. She argues that many programs are so difficult 
to evaluate because they are based on poorly articulated 
assumptions. Weiss emphasizes the need to look at the 
mini-steps if a long term outcome is to be attained. She also 
stresses policy makers to be specific about the theories of 
change that guide their work arguing that this clarity would 
help improve policies and enhance their ability to claim 
credit for outcomes that were predicted in their theory [9]. 
Leeuw [10] lists steps that elicit the theory of change 
underlying a planned program. These enforce the evaluator 
to work with a wide range of stakeholders. These are the 
steps: 

STEP 1: The focus is on long term vision of a program and 
likely to relate to a time scale that lies beyond its timeframe. 
STEP 2: Having agreed the ultimate aim of the program 
stakeholders are encouraged to consider the necessary 
outcomes that will be necessary by the end of the program. 
STEP 3 &STEP 4: Stakeholders are asked to articulate the 
types of outputs and short-term outcomes that will help them 
achieve the specified targets. The theory of change was 
lauded by Weiss as practical and effective to communities 
engaged in transformation efforts. The Organizational 
Research Services [11] succinctly summarises this theory of 


change: 

Every community needs a roadmap for change. Instead of 
bridges, avenues and freeways, this map would illustrate 
destinations of progress and the routes to travel on the way to 
achieving progress. The map would also provide 
commentary about assumptions, such as the final destination, 
the context for the map, the processes to engage in during the 
journey and the belief system that underlies the importance 
of travelling in a particular way. This type of map is called a 
"theory of change”. 

This theory is a clear road map for change, sometimes 
referred to as the logic model it guides those engaged in the 
change process. The theory of change ensures that those 
engaged on a journey to change process do not lose their 
direction. It is unthinkable to contemplate the success of any 
transformation guidelines without clear guidelines: the 
theory of change. One of the useful techniques of the theory 
of change is to employ an outcome map which is a visual 
diagram that spells out relationships between initiative 
strategies and intended results. The theory of change usually 
yields two products: An outcome map and a list of 
assumptions about change [11]. Therefore, Weiss’s theory of 
change is an approach that shows why an initiative works or 
does not [12]. Moreover, the theory of change seeks to 
improve how programs are implemented. Weiss perceives 
this theory and its role in evaluation research. Mapping 
theories of change, the politics of evaluation, theory of 
change for planning evaluation, managing for outcomes and 
logic modelling are some of the topics which are relevant to 
Weiss as she explicates the theory of change. 

3. The Complexity in the Theory of 
Change 

Stame [13] points out that the theory-based evaluations 
have assisted in opening the "black box” of programs. This 
author defines the black box as the space between the actual 
input and the expected output of the program. In a section 
below, the discussion will briefly focus on how Weiss 
perceives the role of politics in evaluation. Weiss points out 
that programs are confused and frequently program theory is 
affected by politics. To illustrate this complexity, Stame [13] 
writes: 

For Weiss programs do have theories although as 
confused as a level of muddling through permits. The black 
box is full of many theories. She calls them ‘theories of 
change’ (Weiss, 1995): they take the form of assumptions, 
tacit understandings, etc.: often more than one for the same 
program. In a remarkable effort (Weiss, 1995:74), she lists 
seven assumptions, and many more sub-assumptions, that 
could underlie a program on the provision of services in 
comprehensive community initiatives. 

Just like her contemporaries such as Chen and Patton, 
Weiss underscored the challenges of evaluating programs. A 
publication of the treasury Board of Canada Secretariat [14] 
points out that over the past two decades the theory-based 
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approaches have moved into the mainstream of thinking and 
practice about how interventions are designed and evaluated. 
These can help evaluators address a variety of challenges, 
such as coming to terms with the inherent complexity of 
certain types of interventions. 

4. The Politics of Evaluation 

Weiss refers to evaluation research as a rational enterprise. 
However, she highlights the political constraints and 
resistances that exist. Weiss [15] argues: 

But evaluation is a rational enterprise that takes place in a 
political context. Political considerations intrude in three 
major ways, and the evaluator who fails to recognize their 
presence is in for a series of shocks and frustrations: First, the 
policies and programs with which evaluation deals are the 
creatures of political decisions. They were proposed, defined, 
debated, enacted, and funded through political processes and 
in implementation they remain subject to pressures - both 
supportive and hostile - that arise out of the play of politics. 

Weiss further adds that evaluation’s reports enter the 
political arena and assume a political stance. Even programs 
that are evaluated are political creatures. “The programs with 
which the evaluator deals are not neutral, antiseptic, 
laboratory-type entities. They emerged from the rough and 
tumble of political support, opposition and bargaining” [15]. 
However, before we explore her theory, it is fitting to briefly 
look at what other evaluation experts say about the influence 
of politics in research. Cohen, Manion and Harrison [16] cite 
MacDonald who states that evaluation is inherently political. 
Furthermore, this author has characterized evaluation as 
having threefold qualities; autocratic, bureaucratic as well as 
democratic. “However, the truth of the matter is far more 
blurred than these distinctions suggest. Two principal causes 
of this blurring lie in the funding and the politics of both 
evaluation and research” [16]. Research is always affected 
by what is happening in society. Weiss takes evaluation as 
step further as she proclaims that evaluations should help in 
decision making. Rowe and Taylor [17] concur as they point 
out: 

Evaluation makes little sense unless it is understood as 
part of a learning process. Learning distinguishes it from 
audit, performance management and reporting. Indeed, for 
some, evaluations should be explicitly framed to ensure their 
use by policymakers and other stakeholders, including 
practitioners (Patton, 1997). Without wishing to engage in 
the methodological implications of such an approach, it is 
common to assume that evaluations of public services will, 
in some way, contribute to a body of knowledge and 
understanding... 

Weiss looks at how the evaluator has to face the task of 
sifting the real from the unreal. Furthermore, she argues that 
the evaluators have to direct their research towards genuine 
goals [15]. As pointed out above, the politics can undermine 
evaluation. Furthermore, Weiss [5] asserts: 

While the evaluation study is in progress, political 


pressured can alter or undermine it. Let us look at one final 
example of how organizational politics can affect the shape 
of evaluation research. Programs do not always keep to their 
original course; over time, often a short span of time, they 
can shift in activities and in overall strategies even in the 
objectives they seek to attain. 

Unfortunately, frequently programs are clouted by politics 
and this makes the work of evaluators very difficult. In the 
above extract, Weiss explains that the political pressures can 
change the course of a program hence one will be certain that 
in turn evaluation will be affected. Costan [18] argues that 
there are scholars who like Weiss explain the political aspect 
in research. Costan points out that on the one hand evaluation 
has a rational analytical school whose scholars state that the 
goals of evaluation are to seek "the truth”. While on the other, 
the responsive school points out that there is no unique 
reality i.e. the scope of evaluation as the quest for truth. “In 
the responsive school tradition, the evaluation forms part of a 
continuous process driven by political and other interests that 
may be leading at bets to some agreement on images of 
realities” [18]. This necessitates the role of the evaluator to 
transform and be that of a negotiation process. Here the 
evaluator’s role is to ensure that consensus is reached; this is 
what political constraints can do to evaluations. In the 
section below the focus is on some of the “hazards” Weiss 
highlights as responsible in making evaluation increasingly 
political [2]. 

5. Relationships with Funding Bodies 

Weiss points out an important supposition when she states 
that evaluation money today is given out with restrictions. 
Usually the government’s request for proposals specify what 
needs to be accomplished by the research, it also specifies 
details of objectives, indicators, timing, analysis and 
reporting. In short, the government agencies want to closely 
monitor the processes of evaluation during the course of the 
study. Some of these agencies would want biweekly 
conferences or monthly reports to ensure that all is still on 
track. “Government agencies may seek only to enforce 
standards of relevance and research quality, but they almost 
inevitably become suspect of political pressure, pressure to 
vindicate the program and justify its budget” [2]. 

6. Relationships with Program 
Personnel 

Few program staff like evaluators who want to measure 
the success or outcomes of a program they are running. 
Program drivers who “know” that their program is doing 
well, the evaluation is unnecessary and if shows few positive 
aspects it becomes a threat to the future of the program. 
These might offer reasons why data given to the evaluators 
may be limited. “Occasionally what program staff deems 
more ‘relevant data’ may be supplied. Even where this is not 
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so, the general atmosphere of uncordiality can dim the 
evaluator’s spirit and his study” [2], 

7. Drawing Recommendations 

Due to the political context of evaluation, the act of 
drawing implications from study data becomes chancier than 
ever. As pointed out above, Weiss refers to evaluation as a 
“black box” studies. The evaluator takes measurements on 
factors relating to program goals before measurement on 
factors relevant to program goals. The evaluator may then 
conclude that the program has succeeded in achieving its 
goals to the observed extent. When exercising the black box 
studies evaluators jump to recommendations without 
providing the necessary data. 

8. Null Results 

The greatest political problem in evaluation is when 
results turn out to be negative. There are a number of 
instances where the evaluator has to relay the results that an 
evaluated program does not show any success. Weiss argues 
though that null evaluation results should not lead to the 
discontinuation of programs but rather to their improvement. 
Evaluators can and should make use of null results. Weiss 
[15] argues: 

For a time, it may be a reasonable response to call 
attention to possible variations that may increase 
success-higher levels of funding, more skilled management, 
better trained staff, better coordination with other services, 
more intensive treatment and so on. If these 
recommendations are ignored, if the political response is to 
persist with the same low cost, low-trouble program, there 
is not much more that the social scientist can learn by 
evaluating participant outcomes. 

Weiss also adds that there needs to be a thorough 
re-examination of the primary problem. Moreover, she 
argues that, “it is important to improve the craft of 
evaluation so that we have greater confidence in its results.” 
Weak measurement reliability, according to Weiss gives 
rise to the null effect. 

9. The Role of the Evaluator 

Luo [19] writes about the role of the evaluator and 
describes it as a fundamental issue for evaluation. Luo [19] 
differentiates between five perspectives on the envisaged 
roles of evaluators. Among these are Weiss’s views on the 
role of the evaluator. Luo [19] points out: Weiss emphasizes 
the evaluator’s special role in promoting the use of his/her 
evaluation results, especially in the policy-making process. 
She is frustrated about the fact that “evaluation results have 
generally not exerted significant influence on program 
decisions, ” and she argues that evaluation should start out 
with the use of evaluation to the natural processes of 


discrimination and application. Weiss claims that evaluation 
“should be continuing education for program manager, 
planners and policy makers ”.... She urges evaluators to look 
beyond the instrumental use of evaluation results and 
conduct “enlightenment” research that “provides evidence 
that can be used by men and women of judgment in their 
efforts to research solutions ”. 

Luo [19] examines the strengths as well as weaknesses in 
Weiss’s theory. He points out that Weiss differentiates 
between the role of the evaluator from the role of a 
researcher by addressing the complex political process 
explored above. Weiss also wants to see evaluators engaged 
in evaluation that can be used in policy-making in the form 
of “enlightenment” rather than instrumental use”. Barbier 
[20] endorses this when he states that the evaluator is not 
merely a technician meeting the client’s needs but is 
expected to use his or her own room for maneuver. And 
again as seen above politics threaten the evaluator’s 
involvement and it can be very difficult for them to avoid 
politics. As pointed out above, Weiss [15] argues that 
evaluations have to take political stances. Evaluators will 
have to live with this reality although Weiss points it out 
clearly that evaluation standard has to favor the public 
general interest at all times. Conley-Tyler [21] highlights 
that most evaluators emphasize the importance of gathering 
knowledge about the program being evaluated so as to 
ensure that the evaluation reflects the program and its 
context. These authors differentiate between internal and 
external evaluators. Internal evaluators tend to understand 
the population served the program under study as well as the 
staff. Furthermore, Conley-Tyler [21] asserts: 

This means that internal evaluators have an advantage 
over external evaluators since they work in the environment 
in which the program operates and may have been personally 
involved in some part of the program planning Weiss 1972, 
p.38). As part of the organization, internal evaluators have 
first-hand knowledge of the organization’s philosophy, 
policies, procedures, personnel and management. 

However, despite this important role of the internal 
evaluator, external evaluators have a huge role to play. Weiss 
[2] stresses that when selecting evaluators, role-players 
should look both at skills and experience. Weiss points out 
that organizations are prone to select academic researchers 
with credentials as they ignore the valuable internal expertise. 
Yet many would prefer people with no stake in the program 
for the sake of objectivity. Internal evaluators are usually 
viewed with skepticism; not being objective or reliable - this 
is what [2] refers to as the “lingering suspicion”. A number 
of authors have underscored the role of evaluation methods 
in determining the effectiveness of evaluation. Skoletset al. 
[22] postulates: 

Some evaluation theorists suggested that traditionally an 
evaluators’ choice of an evaluation method determined 
his/her evaluator role (Greene, 2000; Mark, 2002; Weiss, 
1998). Weiss (1998) asserted that the traditional and still 
dominant role of conceptualization of evaluators is methods 
based and representative of a neutral, detached social 
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scientist: “The traditional role of the inquiry...She put his 
trust in methodology” (p.90). Weiss also noted that role 
conceptualizations of evaluators were beginning to shift 
toward viewing the evaluator as more of a collaborator with 
program staff. 

10. Conclusion 

This paper examined the complexity of evaluation. In her 
theory Weiss illustrates it well that the theory of evaluation 
encompasses a number of aspects that evaluators need to 
understand. A wise evaluator will try and understand the 
various processes involved in evaluation process. Weiss also 
clearly supports programs of planned social change. She also 
adds that if real change is to happen evaluation should follow 
a well-drawn map. Finally, Weiss is also aware that the 
political pressures cannot be avoided during any evaluation 
process. In fact, she maintains that political processes have 
become part of evaluations of programs today. 
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